Skip to content

Conversation

@zanieb
Copy link
Member

@zanieb zanieb commented Jul 9, 2025

GitHub has started enforcing a 1000 file limit, which breaks us.

I first encountered it in CI during the release: https://github.com/astral-sh/python-build-standalone/actions/runs/16156480423

Then got to an actual error message via the CLI:

❯ gh release upload 20250708 dist/cpython-3.14.0b4-riscv64-unknown-linux-gnu-freethreaded+lto-20250708T1536.tar.zst
HTTP 422: Validation Failed (https://uploads.github.com/repos/astral-sh/python-build-standalone/releases/230978017/assets?label=&name=cpython-3.14.0b4-riscv64-unknown-linux-gnu-freethreaded%2Blto-20250708T1536.tar.zst)
file_count limited to 1000 assets per release

You can see they documented a limit recently:

https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases#storage-and-bandwidth-quotas

https://web.archive.org/web/20250629152715/https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases

GitHub has started enforcing a 1000 file limit, which breaks us
@zanieb zanieb added the ci:skip label Jul 9, 2025
Copy link
Collaborator

@indygreg indygreg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh.

I wouldn't be surprised if someone somewhere is automatically downloading the .sha256 file and using it for verification. We could grab the GitHub release artifact download stats to quickly verify impact. But I'm getting API timeouts when trying that :/

Nuking these individual files and guiding people to use the consolidated sha256sums file is likely the path of least resistance to moving forward.

Only other alternative I can think of is to have a GitHub release per Python version of OS or something. But that is super ugly. I'm not aware of any other GitHub project doing that.

@zanieb
Copy link
Member Author

zanieb commented Jul 9, 2025

I verified this resolves the problem by releasing from this branch https://github.com/astral-sh/python-build-standalone/releases/tag/20250708

Unfortunately I think we'll need to move to the consolidated file without warning, since we can't otherwise publish. Even if we did multiple GitHub Releases, that'd be a breaking change for consumers.

I think @geofft had expressed interest in moving away from the individual checksum files.

@zanieb zanieb marked this pull request as ready for review July 9, 2025 00:59
@geofft
Copy link
Collaborator

geofft commented Jul 9, 2025

I'm okay with dropping them and I think the value of it is minimal since it's over HTTPS and from the same web server anyway. Even with 500 artifacts the full SHA256SUMS file is manageable for people who need it.

I'd also like, at some point, to move to our own API that we control, even if it sends redirects to GitHub artifact storage for actual downloads, so things like this aren't forced breaking changes.

@zanieb zanieb merged commit e996271 into main Jul 9, 2025
11 checks passed
@zanieb zanieb deleted the zb/no-sums branch July 9, 2025 12:24
konstin added a commit to astral-sh/uv that referenced this pull request Jul 18, 2025
We recently ran over the file limit and had to drop hash file from the releases page (astral-sh/python-build-standalone#691). Conveniently, GitHub has recently started to add a SHA256 digest to the API. GitHub did not backfill the hashes for the old releases, so use the API hashes for newer assets, while we use our own hash files for older releases.
konstin added a commit to astral-sh/uv that referenced this pull request Jul 18, 2025
We recently ran over the file limit and had to drop hash file from the
releases page in favor of bulk SHA256SUMS files
(astral-sh/python-build-standalone#691).
Conveniently, GitHub has recently started to add a SHA256 digest to the
API. GitHub did not backfill the hashes for the old releases, so use the
API hashes for newer assets, and eventually only download SHA256SUMS for
older releases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants